75 research outputs found

    An Efficient Algorithm for Deep Stochastic Contextual Bandits

    Full text link
    In stochastic contextual bandit (SCB) problems, an agent selects an action based on certain observed context to maximize the cumulative reward over iterations. Recently there have been a few studies using a deep neural network (DNN) to predict the expected reward for an action, and the DNN is trained by a stochastic gradient based method. However, convergence analysis has been greatly ignored to examine whether and where these methods converge. In this work, we formulate the SCB that uses a DNN reward function as a non-convex stochastic optimization problem, and design a stage-wise stochastic gradient descent algorithm to optimize the problem and determine the action policy. We prove that with high probability, the action sequence chosen by this algorithm converges to a greedy action policy respecting a local optimal reward function. Extensive experiments have been performed to demonstrate the effectiveness and efficiency of the proposed algorithm on multiple real-world datasets.Comment: Accepted by AAAI 202

    A novel routing optimization strategy based on reinforcement learning in perception layer networks

    Get PDF
    Wireless sensor networks have become incredibly popular due to the Internet of Thingsā€™ (IoT) rapid development. IoT routing is the basis for the efficient operation of the perception-layer network. As a popular type of machine learning, reinforcement learning techniques have gained significant attention due to their successful application in the field of network communication. In the traditional Routing Protocol for low-power and Lossy Networks (RPL) protocol, to solve the fairness of control message transmission between IoT terminals, a fair broadcast suppression mechanism, or Drizzle algorithm, is usually used, but the Drizzle algorithm cannot allocate priority. Moreover, the Drizzle algorithm keeps changing its redundant constant k value but never converges to the optimal value of k. To address this problem, this paper uses a combination based on reinforcement learning (RL) and trickle timer. This paper proposes an RL Intelligent Adaptive Trickle-Timer Algorithm (RLATT) for routing optimization of the IoT awareness layer. RLATT has triple-optimized the trickle timer algorithm. To verify the algorithmā€™s effectiveness, the simulation is carried out on Contiki operating system and compared with the standard trickling timer and Drizzle algorithm. Experiments show that the proposed algorithm performs better in terms of packet delivery ratio (PDR), power consumption, network convergence time, and total control cost ratio

    Infrastructure for mobile sensor network in the Singapore coastal zone

    Get PDF
    URL to conference page. Scroll down to 2010 conference, click on "Paper and session list," and search the PDF for Patrikalakis.Singapore is an island nation located at southern tip of the Malaysian Peninsula. She is at a strategic location along major shipping routes and therefore has one of the busiest harbors in the world. Having a safe and secure harbor environment is vital to maintain trade and growth in the country and region. To help build and maintain a safe harbor environment, the Center of Environmental Sensing and Modelling (CENSAM) under the Singapore-MIT Alliance for Research and Technology (SMART) is developing a mobile sensor network in the Singapore coastal zone

    Identification of Competing Endogenous RNA Regulatory Networks in Vitamin A Deficiency-Induced Congenital Scoliosis by Transcriptome Sequencing Analysis

    Get PDF
    Background/Aims: Congenital scoliosis (CS) is a result of anomalous development of vertebrae and is frequently associated with somitogenesis malformation. Although noncoding RNAs (ncRNAs) have been recently determined to be involved in the pathogenesis of CS, the competing endogenous RNA (ceRNA) regulatory networks in CS remain largely unknown. Methods: Sequencing was conducted to explore the ncRNA expression profiles in rat embryos (gestation day 9) following vitamin A deficiency (VAD) (n = 9 for the vitamin A deficiency-induced congenital scoliosis (VAD-CS) group and n = 4 for the control group). Real-time reverse transcriptase polymerase chain reaction (RT-PCR) was conducted to verify the expression levels of selected mRNAs, long noncoding RNAs (lncRNAs), circular RNAs (circRNAs), and microRNAs (miRNAs). Bioinformatics analysis was used to discover the possible relationships and functions of the ceRNAs. Results: A total of 749 mRNAs, 56 miRNAs, 685 lncRNAs, and 70 circRNAs were identified to have significantly different expression levels in the two groups. Wnt, PI3K-ATK, FoxO, EGFR, and mTOR were found to be the most significant pathways involved in VAD-CS pathogenesis. The circRNA/miRNA/mRNA and lncRNA/miRNA/mRNA networks of CS were built, and the gene expression mechanisms regulated by ncRNAs were unveiled via the ceRNA regulatory networks. Conclusion: We comprehensively identified ceRNA regulatory networks of embryonic somite development in VAD-CS as well as revealed the contribution of different ncRNA expression profiles. Our data demonstrate the association between mRNAs and ncRNAs in the pathogenic mechanism of CS

    Room-temperature quantum interference in single perovskite quantum dot junctions

    Get PDF
    钙钛ēŸæꝐꖙē”±äŗŽå…¶é«˜é‡å­äŗ§ēŽ‡ć€č½½ęµå­čæē§»ēŽ‡å’Œē‹¬ē‰¹ēš„å…‰č‡“发光ē‰¹ę€§č€ŒåœØ光ē”µęę–™é¢†åŸŸå­˜åœØčÆøå¤šę½œåœØēš„é‡č¦åŗ”ē”Ø怂ē ”ē©¶é’™é’›ēŸæꝐꖙåœØēŗ³ē±³å°ŗåŗ¦äø‹ē”µč·č¾“čæēš„ē‹¬ē‰¹å°ŗåÆøꕈåŗ”åƹ钙钛ēŸæ光ē”µå™Ø件ēš„č®¾č®”å’Œå¼€å‘å…·ęœ‰é‡č¦ēš„ęŒ‡åÆ¼ę„ä¹‰ć€‚ę“Ŗę–‡ę™¶ę•™ęŽˆčƾ题ē»„åŸŗäŗŽęœŗę¢°åÆęŽ§č£‚ē»“ꊀęœÆč‡Ŗäø»ē ”发äŗ†å…·ęœ‰ēš®ē±³ēŗ§ä½ē§»č°ƒęŽ§ēµę•åŗ¦å’Œé£žå®‰ēŗ§ē”µå­¦ęµ‹é‡ē²¾åŗ¦ēš„ē²¾åƆē§‘å­¦ä»Ŗå™Øļ¼ŒåÆ¹å—å¼€å¤§å­¦ęŽč·ƒé¾™å‰Æę•™ęŽˆå›¢é˜Ÿåˆęˆēš„钙钛ēŸæ量子ē‚¹čæ›č”Œäŗ†ę·±å…„č”Ø征ļ¼Œē ”ē©¶å·„ä½œęˆåŠŸå°†é‡å­å¹²ę¶‰ēš„ē ”ē©¶ä½“ē³»ę‹“å±•č‡³åœØ光ē”µé¢†åŸŸå…·ęœ‰é‡č¦åŗ”ē”Øēš„钙钛ēŸæęę–™é¢†åŸŸļ¼ŒäøŗęœŖę„åˆ¶å¤‡åŸŗäŗŽé‡å­å¹²ę¶‰ę•ˆåŗ”ēš„ę–°åž‹é’™é’›ēŸæå™Øä»¶ęä¾›äŗ†äø€ē§å…Øꖰēš„ę€č·Æ怂 čæ™äø€č·Øå­¦ē§‘国际合作ē ”ē©¶å·„作ę˜ÆåœØ化学化巄学院ę“Ŗę–‡ę™¶ę•™ęŽˆć€č‹±å›½Lancaster 大学ē‰©ē†ē³»Colin J. Lambertę•™ęŽˆä»„åŠå—å¼€å¤§å­¦ē”µå­äæ”ęÆäøŽå…‰ē”µå·„ēØ‹å­¦é™¢ęŽč·ƒé¾™å‰Æꕙꎈēš„å…±åŒęŒ‡åƼäø‹å®Œęˆēš„ć€‚åŒ–å·„ē³»ē”•å£«ē ”ē©¶ē”Ÿéƒ‘ęµ·å®ć€Lancaster University大学Songjun Houåšå£«ć€å—å¼€å¤§å­¦ē”•å£«ē ”ē©¶ē”Ÿč¾›ę™Ø光äøŗč®ŗꖇē¬¬äø€ä½œč€…ć€‚åšå£«åŽęž—ē¦„ę˜„ļ¼Œåšå£«ē ”ē©¶ē”Ÿč°­åæ—å†°ć€éƒ‘ēå©·ļ¼Œē”•å£«ē ”ē©¶ē”Ÿč’‹ęž«ć€å¼ ē‘ę¼Ŗļ¼Œęœ¬ē§‘ē”Ÿä½•ę–‡ēæ”ć€ęŽåŗ†ę°‘ē­‰å‚äøŽäŗ†č®ŗꖇēš„ē ”ē©¶å·„ä½œć€‚åˆ˜äæŠę‰¬ē‰¹ä»»å‰Æē ”ē©¶å‘˜ć€åøˆä½³å‰Æę•™ęŽˆå’ŒčØęœ¬ę ‹å¾®ēŗ³ē±³ē ”ē©¶é™¢ęØꉬå‰Æę•™ęŽˆä¹Ÿå‚äøŽäŗ†éƒØåˆ†ęŒ‡åÆ¼å·„ä½œć€‚The studies of quantum interference effects through bulk perovskite materials at the ƅngstrom scale still remain as a major challenge. Herein, we provide the observation of roomtemperature quantum interference effects in metal halide perovskite quantum dots (QDs) using the mechanically controllable break junction technique. Single-QD conductance measurements reveal that there are multiple conductance peaks for the CH3NH3PbBr3 and CH3NH3PbBr2.15Cl0.85 QDs, whose displacement distributions match the lattice constant of QDs, suggesting that the gold electrodes slide through different lattice sites of the QD via Auhalogen coupling. We also observe a distinct conductance ā€˜jumpā€™ at the end of the sliding process, which is further evidence that quantum interference effects dominate charge transport in these single-QD junctions. This conductance ā€˜jumpā€™ is also confirmed by our theoretical calculations utilizing density functional theory combined with quantum transport theory. Our measurements and theory create a pathway to exploit quantum interference effects in quantum-controlled perovskite materials.This work was supported by the National Key R&D Program of China (2017YFA0204902, 2014DFE60170, 2018YFB1500105), the National Natural Science Foundation of China (Nos. 21673195, 21503179, 21490573, 61674084, 61874167), the Open Fund of the Key Laboratory of Optical Information Science & Technology (Nankai University) of China, the Fundamental Research Funds for the Central Universities of China (63181321, 63191414, 96173224), and the 111 Project (B16027), the Tianjin Natural Science Foundation (17JCYBJC41400), FET Open project 767187ā€”QuIET, the EU project BAC-TO-FUEL and the UK EPSRC projects EP/N017188/1, EP/M014452/1. čƄ巄作得到国家重ē‚¹ē ”å‘č®”划čƾ题ļ¼ˆ2017YFA0204902ļ¼‰ć€å›½å®¶č‡Ŗē„¶ē§‘å­¦åŸŗ金ļ¼ˆ21673195态21503179态21490573ļ¼‰ć€åŽ¦é—Ø大学ā€œäŗŗå·„ę™ŗčƒ½åˆ†ęžå¼•ę“Žā€åŒäø€ęµé‡å¤§äø“锹ē­‰é”¹ē›®ēš„čµ„åŠ©ļ¼Œä¹Ÿå¾—到äŗ†å›ŗ体č”Ø面ē‰©ē†åŒ–学国家重ē‚¹å®žéŖŒå®¤ć€čƒ½ęŗęę–™åŒ–å­¦ååŒåˆ›ę–°äø­åæƒēš„ę”Æꌁ

    Room-temperature quantum interference in single perovskite quantum dot junctions.

    Get PDF
    The studies of quantum interference effects through bulk perovskite materials at the ƅngstrom scale still remain as a major challenge. Herein, we provide the observation of room-temperature quantum interference effects in metal halide perovskite quantum dots (QDs) using the mechanically controllable break junction technique. Single-QD conductance measurements reveal that there are multiple conductance peaks for the CH3NH3PbBr3 and CH3NH3PbBr2.15Cl0.85 QDs, whose displacement distributions match the lattice constant of QDs, suggesting that the gold electrodes slide through different lattice sites of the QD via Au-halogen coupling. We also observe a distinct conductance ā€™jumpā€™ at the end of the sliding process, which is further evidence that quantum interference effects dominate charge transport in these single-QD junctions. This conductance ā€™jumpā€™ is also confirmed by our theoretical calculations utilizing density functional theory combined with quantum transport theory. Our measurements and theory create a pathway to exploit quantum interference effects in quantum-controlled perovskite materials

    Room-temperature quantum interference in single perovskite quantum dot junctions

    Get PDF
    The studies of quantum interference effects through bulk perovskite materials at the ƅngstrom scale still remain as a major challenge. Herein, we provide the observation of room-temperature quantum interference effects in metal halide perovskite quantum dots (QDs) using the mechanically controllable break junction technique. Single-QD conductance measurements reveal that there are multiple conductance peaks for the CH3NH3PbBr3 and CH3NH3PbBr2.15Cl0.85 QDs, whose displacement distributions match the lattice constant of QDs, suggesting that the gold electrodes slide through different lattice sites of the QD via Au-halogen coupling. We also observe a distinct conductance ā€˜jumpā€™ at the end of the sliding process, which is further evidence that quantum interference effects dominate charge transport in these single-QD junctions. This conductance ā€˜jumpā€™ is also confirmed by our theoretical calculations utilizing density functional theory combined with quantum transport theory. Our measurements and theory create a pathway to exploit quantum interference effects in quantum-controlled perovskite materials

    Assessing the human immune system through blood transcriptomics

    Get PDF
    Blood is the pipeline of the immune system. Assessing changes in transcript abundance in blood on a genome-wide scale affords a comprehensive view of the status of the immune system in health and disease. This review summarizes the work that has used this approach to identify therapeutic targets and biomarker signatures in the field of autoimmunity and infectious disease. Recent technological and methodological advances that will carry the blood transcriptome research field forward are also discussed
    • ā€¦
    corecore